A Workload-Adaptive Streaming Partitioner for Distributed Graph Stores

Abstract Streaming graph partitioning methods have recently gained attention due to their ability scale very large graphs with limited resources. However, many such do not consider workload and characteristics. This may degrade the performance of queries by increasing inter-node communication computational load imbalance. Moreover, existing workload-aware cannot consistently provide good as they dynamic workloads that keep emerging in applications. We address these issues proposing a novel workload-adaptive streaming partitioner named WASP, aims achieve low-latency high-throughput online queries. As each typically contains frequent query patterns, WASP exploits capture active vertices edges which are frequently visited traversed, respectively. information is used heuristically improve quality partitions either avoiding concentration few proportional visit frequencies or reducing probability cut traversal frequencies. In order assess impact on store show how easily approach can be plugged top system, we exploit it distributed graph-based RDF store. Our experiments over three synthetic real-world datasets corresponding static achieves better against state-of-the-art partitioners, especially workloads.